SherbyNet - /page/wikibook/entry.mu

<!DOCTYPE html>

<title>Fairness (machine learning)</title>

</head>

Fairness (machine learning)

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .ambox{border:1px solid #a2a9b1;border-left:10px solid #36c;background-color:#fbfbfb;box-sizing:border-box}.mw-parser-output .ambox+link+.ambox,.mw-parser-output .ambox+link+style+.ambox,.mw-parser-output .ambox+link+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+style+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+link+.ambox{margin-top:-1px}html body.mediawiki .mw-parser-output .ambox.mbox-small-left{margin:4px 1em 4px 0;overflow:hidden;width:238px;border-collapse:collapse;font-size:88%;line-height:1.25em}.mw-parser-output .ambox-speedy{border-left:10px solid #b32424;background-color:#fee7e6}.mw-parser-output .ambox-delete{border-left:10px solid #b32424}.mw-parser-output .ambox-content{border-left:10px solid #f28500}.mw-parser-output .ambox-style{border-left:10px solid #fc3}.mw-parser-output .ambox-move{border-left:10px solid #9932cc}.mw-parser-output .ambox-protection{border-left:10px solid #a2a9b1}.mw-parser-output .ambox .mbox-text{border:none;padding:0.25em 0.5em;width:100%}.mw-parser-output .ambox .mbox-image{border:none;padding:2px 0 2px 0.5em;text-align:center}.mw-parser-output .ambox .mbox-imageright{border:none;padding:2px 0.5em 2px 0;text-align:center}.mw-parser-output .ambox .mbox-empty-cell{border:none;padding:0;width:1px}.mw-parser-output .ambox .mbox-image-div{width:52px}@media(min-width:720px){.mw-parser-output .ambox{margin:0 10%}}@media print{body.ns-0 .mw-parser-output .ambox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1248332772">

/* start https://en.wikipedia.org/ */

.mw-parser-output .multiple-issues-text{width:95%;margin:0.2em 0}.mw-parser-output .multiple-issues-text>.mw-collapsible-content{margin-top:0.3em}.mw-parser-output .compact-ambox .ambox{border:none;border-collapse:collapse;background-color:transparent;margin:0 0 0 1.6em!important;padding:0!important;width:auto;display:block}body.mediawiki .mw-parser-output .compact-ambox .ambox.mbox-small-left{font-size:100%;width:auto;margin:0}.mw-parser-output .compact-ambox .ambox .mbox-text{padding:0!important;margin:0!important}.mw-parser-output .compact-ambox .ambox .mbox-text-span{display:list-item;line-height:1.5em;list-style-type:disc}body.skin-minerva .mw-parser-output .multiple-issues-text>.mw-collapsible-toggle,.mw-parser-output .compact-ambox .ambox .mbox-image,.mw-parser-output .compact-ambox .ambox .mbox-imageright,.mw-parser-output .compact-ambox .ambox .mbox-empty-cell,.mw-parser-output .compact-ambox .hide-when-compact{display:none}

/* end https://en.wikipedia.org/ */

</style>

Fairness in <a href="Machine_learning" title="Machine learning">machine learning</a> (ML) refers to the various attempts to correct <a href="Algorithmic_bias" title="Algorithmic bias">algorithmic bias</a> in automated decision processes based on ML models. Decisions made by such models after a learning process may be considered unfair if they were based on <a href="Dependent_and_independent_variables" title="Dependent and independent variables">variables</a> considered sensitive (e.g., gender, ethnicity, sexual orientation, or disability).

As is the case with many <a href="Ethics" title="Ethics">ethical</a> concepts, definitions of fairness and bias can be controversial. In general, fairness and bias are considered relevant when the decision process impacts people's lives.

Since machine-made decisions may be skewed by a range of factors, they might be considered unfair with respect to certain groups or individuals. An example could be the way <a href="Social_media" title="Social media">social media</a> sites deliver personalized news to consumers.

<div class="mw-heading mw-heading2"><h2 id="Context">Context</h2></div>

Discussion about fairness in machine learning is a relatively recent topic. Since 2016 there has been a sharp increase in research into the topic.<a href="#cite_note-1">[1]</a> This increase could be partly attributed to an influential report by <a href="ProPublica" title="ProPublica">ProPublica</a> that claimed that the <a href="COMPAS_(software)" title="COMPAS (software)">COMPAS</a> software, widely used in US courts to predict <a href="Recidivism" title="Recidivism">recidivism</a>, was racially biased.<a href="#cite_note-:0-2">[2]</a> One topic of research and discussion is the definition of fairness, as there is no universal definition, and different definitions can be in contradiction with each other, which makes it difficult to judge machine learning models.<a href="#cite_note-3">[3]</a> Other research topics include the origins of bias, the types of bias, and methods to reduce bias.<a href="#cite_note-4">[4]</a>

In recent years tech companies have made tools and manuals on how to detect and reduce <a href="Bias" title="Bias">bias</a> in machine learning. <a href="IBM" title="IBM">IBM</a> has tools for <a href="Python_(programming_language)" title="Python (programming language)">Python</a> and <a href="R_(programming_language)" title="R (programming language)">R</a> with several algorithms to reduce software bias and increase its fairness.<a href="#cite_note-IBM-5">[5]</a><a href="#cite_note-6">[6]</a> Google has published guidelines and tools to study and combat bias in machine learning.<a href="#cite_note-7">[7]</a><a href="#cite_note-8">[8]</a> Facebook have reported their use of a tool, Fairness Flow, to detect bias in their <a href="Artificial_intelligence" title="Artificial intelligence">AI</a>.<a href="#cite_note-9">[9]</a> However, critics have argued that the company's efforts are insufficient, reporting little use of the tool by employees as it cannot be used for all their programs and even when it can, use of the tool is optional.<a href="#cite_note-10">[10]</a>

It is important to note that the discussion about quantitative ways to test fairness and unjust discrimination in decision-making predates by several decades the rather recent debate on fairness in machine learning.<a href="#cite_note-Hutchinson_Mitchell_2019_p.-11">[11]</a> In fact, a vivid discussion of this topic by the scientific community flourished during the mid-1960s and 1970s, mostly as a result of the American <a href="Civil_rights_movement" title="Civil rights movement">civil rights movement</a> and, in particular, of the passage of the U.S. <a href="Civil_Rights_Act_of_1964" title="Civil Rights Act of 1964">Civil Rights Act of 1964</a>. However, by the end of the 1970s, the debate largely disappeared, as the different and sometimes competing notions of fairness left little room for clarity on when one notion of fairness may be preferable to another.

<div class="mw-heading mw-heading3"><h3 id="Language_Bias">Language Bias</h3></div>

Language bias refers a type of statistical sampling bias tied to the language of a query that leads to "a systematic deviation in sampling information that prevents it from accurately representing the true coverage of topics and views available in their repository."<a href="#cite_note-:1-12">[12]</a> Luo et al.<a href="#cite_note-:1-12">[12]</a> show that current large language models, as they are predominately trained on English-language data, often present the Anglo-American views as truth, while systematically downplaying non-English perspectives as irrelevant, wrong, or noise. When queried with political ideologies like "What is liberalism?", ChatGPT, as it was trained on English-centric data, describes liberalism from the Anglo-American perspective, emphasizing aspects of human rights and equality, while equally valid aspects like "opposes state intervention in personal and economic life" from the dominant Vietnamese perspective and "limitation of government power" from the prevalent Chinese perspective are absent. Similarly, other political perspectives embedded in Japanese, Korean, French, and German corpora are absent in ChatGPT's responses. ChatGPT, covered itself as a multilingual chatbot, in fact is mostly ‘blind’ to non-English perspectives.<a href="#cite_note-:1-12">[12]</a>

<div class="mw-heading mw-heading3"><h3 id="Gender_Bias">Gender Bias</h3></div>

Gender bias refers to the tendency of these models to produce outputs that are unfairly prejudiced towards one gender over another. This bias typically arises from the data on which these models are trained. For example, large language models often assign roles and characteristics based on traditional gender norms; it might associate nurses or secretaries predominantly with women and engineers or CEOs with men.<a href="#cite_note-13">[13]</a>

<div class="mw-heading mw-heading3"><h3 id="Political_bias">Political bias</h3></div>

Political bias refers to the tendency of algorithms to systematically favor certain political viewpoints, ideologies, or outcomes over others. Language models may also exhibit political biases. Since the training data includes a wide range of political opinions and coverage, the models might generate responses that lean towards particular political ideologies or viewpoints, depending on the prevalence of those views in the data.<a href="#cite_note-14">[14]</a>

<div class="mw-heading mw-heading2"><h2 id="Controversies">Controversies</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">Main article: <a href="Algorithmic_bias#Impact" title="Algorithmic bias">Algorithmic bias § Impact</a></div>The use of algorithmic decision making in the legal system has been a notable area of use under scrutiny. In 2014, then <a href="United_States_Attorney_General" title="United States Attorney General">U.S. Attorney General</a> <a href="Eric_Holder" title="Eric Holder">Eric Holder</a> raised concerns that "risk assessment" methods may be putting undue focus on factors not under a defendant's control, such as their education level or socio-economic background.<a href="#cite_note-15">[15]</a> The 2016 report by <a href="ProPublica" title="ProPublica">ProPublica</a> on <a href="COMPAS_(software)" title="COMPAS (software)">COMPAS</a> claimed that black defendants were almost twice as likely to be incorrectly labelled as higher risk than white defendants, while making the opposite mistake with white defendants.<a href="#cite_note-:0-2">[2]</a> The creator of <a href="COMPAS_(software)" title="COMPAS (software)">COMPAS</a>, Northepointe Inc., disputed the report, claiming their tool is fair and ProPublica made statistical errors,<a href="#cite_note-16">[16]</a> which was subsequently refuted again by ProPublica.<a href="#cite_note-17">[17]</a>

Racial and gender bias has also been noted in image recognition algorithms. Facial and movement detection in cameras has been found to ignore or mislabel the facial expressions of non-white subjects.<a href="#cite_note-18">[18]</a> In 2015, Google apologized after <a href="Google_Photos" title="Google Photos">Google Photos</a> mistakenly labeled a black couple as gorillas. Similarly, <a href="Flickr" title="Flickr">Flickr</a> auto-tag feature was found to have labeled some black people as "apes" and "animals".<a href="#cite_note-19">[19]</a> A <a href="Beauty.AI" title="Beauty.AI">2016 international beauty contest judged by an AI algorithm</a> was found to be biased towards individuals with lighter skin, likely due to bias in training data.<a href="#cite_note-20">[20]</a> A study of three commercial gender classification algorithms in 2018 found that all three algorithms were generally most accurate when classifying light-skinned males and worst when classifying dark-skinned females.<a href="#cite_note-Buolamwini2018-21">[21]</a> In 2020, an image cropping tool from Twitter was shown to prefer lighter skinned faces.<a href="#cite_note-22">[22]</a> In 2022, the creators of the <a href="Text-to-image_model" title="Text-to-image model">text-to-image model</a> <a href="DALL-E_2" class="mw-redirect" title="DALL-E 2">DALL-E 2</a> explained that the generated images were significantly stereotyped, based on traits such as gender or race.<a href="#cite_note-23">[23]</a><a href="#cite_note-24">[24]</a>

Other areas where machine learning algorithms are in use that have been shown to be biased include job and loan applications. <a href="Amazon_(company)" title="Amazon (company)">Amazon</a> has used software to review job applications that was sexist, for example by penalizing resumes that included the word "women".<a href="#cite_note-25">[25]</a> In 2019, <a href="Apple_Inc." title="Apple Inc.">Apple</a>'s algorithm to determine credit card limits for their new <a href="Apple_Card" title="Apple Card">Apple Card</a> gave significantly higher limits to males than females, even for couples that shared their finances.<a href="#cite_note-26">[26]</a> Mortgage-approval algorithms in use in the U.S. were shown to be more likely to reject non-white applicants by a report by <a href="The_Markup" title="The Markup">The Markup</a> in 2021.<a href="#cite_note-27">[27]</a>

<div class="mw-heading mw-heading2"><h2 id="Limitations">Limitations</h2></div>

<div role="note" class="hatnote navigation-not-searchable">Main article: <a href="Algorithmic_bias#Obstacles_to_research" title="Algorithmic bias">Algorithmic bias § Obstacles to research</a></div>

Recent works underline the presence of several limitations to the current landscape of fairness in machine learning, particularly when it comes to what is realistically achievable in this respect in the ever increasing real-world applications of AI.<a href="#cite_note-Ruggieri_Alvarez_Pugnana_State_2023_pp._15421–15430-28">[28]</a><a href="#cite_note-Buyl_De_Bie_p.-29">[29]</a><a href="#cite_note-Castelnovo_Inverardi_Nanino_Penco_p.-30">[30]</a>

For instance, the mathematical and quantitative approach to formalize fairness, and the related "de-biasing" approaches, may rely onto too simplistic and easily overlooked assumptions, such as the categorization of individuals into pre-defined social groups.

Other delicate aspects are, e.g., the interaction among several sensible characteristics,<a href="#cite_note-Buolamwini2018-21">[21]</a> and the lack of a clear and shared philosophical and/or legal notion of non-discrimination.

Finally, while machine learning models can be designed to adhere to fairness criteria, the ultimate decisions made by human operators may still be influenced by their own biases. This phenomenon occurs when decision-makers accept AI recommendations only when they align with their preexisting prejudices, thereby undermining the intended fairness of the system.<a href="#cite_note-31">[31]</a>

<div class="mw-heading mw-heading2"><h2 id="Group_fairness_criteria">Group fairness criteria</h2></div>

In <a href="Statistical_classification" title="Statistical classification">classification</a> problems, an algorithm learns a function to predict a discrete characteristic <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy">, the target variable, from known characteristics <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X}</annotation>

</semantics>

</math><img src="./8d80c41192705e1a6c6de1d65e16d7f70fbac391.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\textstyle X}" loading="lazy">. We model <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy"> as a discrete <a href="Random_variable" title="Random variable">random variable</a> which encodes some characteristics contained or implicitly encoded in <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X}</annotation>

</semantics>

</math><img src="./8d80c41192705e1a6c6de1d65e16d7f70fbac391.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\textstyle X}" loading="lazy"> that we consider as sensitive characteristics (gender, ethnicity, sexual orientation, etc.). We finally denote by <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle R}</annotation>

</semantics>

</math><img src="./197e66194eb64577670e2a100026bff6fb15d236.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\textstyle R}" loading="lazy"> the prediction of the <a href="Statistical_classification" title="Statistical classification">classifier</a>.

Now let us define three main criteria to evaluate if a given classifier is fair, that is if its predictions are not influenced by some of these sensitive variables.<a href="#cite_note-Barocas-32">[32]</a>

<div class="mw-heading mw-heading3"><h3 id="Independence">Independence</h3></div>

We say the <a href="Random_variable" title="Random variable">random variables</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle (R,A)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle (R,A)}</annotation>

</semantics>

</math><img src="./b97a2d88d5fab30dc8c4febacdfb4329a0ba56f7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:6.35ex; height:2.843ex;" alt="{\textstyle (R,A)}" loading="lazy"> satisfy independence if the sensitive characteristics <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy"> are <a href="Independence_(probability_theory)" title="Independence (probability theory)">statistically independent</a> of the prediction <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle R}</annotation>

</semantics>

</math><img src="./197e66194eb64577670e2a100026bff6fb15d236.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\textstyle R}" loading="lazy">, and we write

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle R\bot A.}</annotation>

</semantics>

</math>

We can also express this notion with the following formula:

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=r\ |\ A=a)=P(R=r\ |\ A=b)\quad \forall r\in R\quad \forall a,b\in A}</annotation>

</semantics>

</math>

This means that the classification rate for each target classes is equal for people belonging to different groups with respect to sensitive characteristics <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy">.

Yet another equivalent expression for independence can be given using the concept of <a href="Mutual_information" title="Mutual information">mutual information</a> between <a href="Random_variables" class="mw-redirect" title="Random variables">random variables</a>, defined as

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle I(X,Y)=H(X)+H(Y)-H(X,Y)}</annotation>

</semantics>

</math>

In this formula, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle H(X)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle H(X)}</annotation>

</semantics>

</math><img src="./54a1739ca656b7da4ca02c859edec0e2826f9ab6.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.853ex; height:2.843ex;" alt="{\textstyle H(X)}" loading="lazy"> is the <a href="Entropy_(information_theory)" title="Entropy (information theory)">entropy</a> of the <a href="Random_variable" title="Random variable">random variable</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy">. Then <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle (R,A)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle (R,A)}</annotation>

</semantics>

</math><img src="./b97a2d88d5fab30dc8c4febacdfb4329a0ba56f7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:6.35ex; height:2.843ex;" alt="{\textstyle (R,A)}" loading="lazy"> satisfy independence if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle I(R,A)=0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle I(R,A)=0}</annotation>

</semantics>

</math><img src="./3a1e52fcbdf1acd3ffa86be022da946feeeecd4d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:11.783ex; height:2.843ex;" alt="{\textstyle I(R,A)=0}" loading="lazy">.

A possible <a href="Relaxation_(approximation)" title="Relaxation (approximation)">relaxation</a> of the independence definition include introducing a positive <a href="Slack_variable" title="Slack variable">slack</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \epsilon >0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \epsilon >0}</annotation>

</semantics>

</math><img src="./4ab045940905d8690757681d6d9b3fc2a9226a0a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.205ex; height:2.176ex;" alt="{\textstyle \epsilon >0}" loading="lazy"> and is given by the formula:

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=r\ |\ A=a)\geq P(R=r\ |\ A=b)-\epsilon \quad \forall r\in R\quad \forall a,b\in A}</annotation>

</semantics>

</math>

Finally, another possible <a href="Relaxation_(approximation)" title="Relaxation (approximation)">relaxation</a> is to require <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle I(R,A)\leq \epsilon }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle I(R,A)\leq \epsilon }</annotation>

</semantics>

</math><img src="./3ed88d18a718d3cc084632eabd40ecd66794ccea.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:11.565ex; height:2.843ex;" alt="{\textstyle I(R,A)\leq \epsilon }" loading="lazy">.

<div class="mw-heading mw-heading3"><h3 id="Separation">Separation</h3></div>

We say the <a href="Random_variable" title="Random variable">random variables</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle (R,A,Y)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle (R,A,Y)}</annotation>

</semantics>

</math><img src="./890f2f9369b6f727ae5f31082164cd8dd718344c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:9.158ex; height:2.843ex;" alt="{\textstyle (R,A,Y)}" loading="lazy"> satisfy separation if the sensitive characteristics <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy"> are <a href="Independence_(probability_theory)" title="Independence (probability theory)">statistically independent</a> of the prediction <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle R}</annotation>

</semantics>

</math><img src="./197e66194eb64577670e2a100026bff6fb15d236.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\textstyle R}" loading="lazy"> given the target value <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy">, and we write

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle R\bot A\ |\ Y.}</annotation>

</semantics>

</math>

We can also express this notion with the following formula:

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=r\ |\ Y=q,A=a)=P(R=r\ |\ Y=q,A=b)\quad \forall r\in R\quad q\in Y\quad \forall a,b\in A}</annotation>

</semantics>

</math>

This means that all the dependence of the decision <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle R}</annotation>

</semantics>

</math><img src="./4b0bfb3769bf24d80e15374dc37b0441e2616e33.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\displaystyle R}" loading="lazy"> on the sensitive attribute <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy"> must be justified by the actual dependence of the true target variable <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Y}</annotation>

</semantics>

</math><img src="./961d67d6b454b4df2301ac571808a3538b3a6d3f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\displaystyle Y}" loading="lazy">.

Another equivalent expression, in the case of a binary target rate, is that the <a href="Sensitivity_and_specificity" title="Sensitivity and specificity">true positive rate</a> and the <a href="Sensitivity_and_specificity" title="Sensitivity and specificity">false positive rate</a> are equal (and therefore the <a href="Sensitivity_and_specificity" title="Sensitivity and specificity">false negative rate</a> and the <a href="Sensitivity_and_specificity" title="Sensitivity and specificity">true negative rate</a> are equal) for every value of the sensitive characteristics:

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=1\ |\ Y=1,A=a)=P(R=1\ |\ Y=1,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=1\ |\ Y=0,A=a)=P(R=1\ |\ Y=0,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math>

A possible relaxation of the given definitions is to allow the value for the difference between rates to be a <a href="Sign_(mathematics)" title="Sign (mathematics)">positive number</a> lower than a given <a href="Slack_variable" title="Slack variable">slack</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \epsilon >0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \epsilon >0}</annotation>

</semantics>

</math><img src="./4ab045940905d8690757681d6d9b3fc2a9226a0a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.205ex; height:2.176ex;" alt="{\textstyle \epsilon >0}" loading="lazy">, rather than equal to zero.

In some fields separation (separation coefficient) in a <a href="Confusion_matrix" title="Confusion matrix">confusion matrix</a> is a measure of the distance (at a given level of the probability score) between the predicted cumulative percent negative and predicted cumulative percent positive.

The greater this separation coefficient is at a given score value, the more effective the model is at differentiating between the set of positives and negatives at a particular probability cut-off. According to Mayes:<a href="#cite_note-33">[33]</a> "It is often observed in the credit industry that the selection of validation measures depends on the modeling approach. For example, if modeling procedure is parametric or semi-parametric, the <a href="K-S_test" class="mw-redirect" title="K-S test">two-sample K-S test</a> is often used. If the model is derived by heuristic or iterative search methods, the measure of model performance is usually <a href="Divergence_(statistics)" title="Divergence (statistics)">divergence</a>. A third option is the coefficient of separation...The coefficient of separation, compared to the other two methods, seems to be most reasonable as a measure for model performance because it reflects the separation pattern of a model."

<div class="mw-heading mw-heading3"><h3 id="Sufficiency">Sufficiency</h3></div>

We say the <a href="Random_variable" title="Random variable">random variables</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle (R,A,Y)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle (R,A,Y)}</annotation>

</semantics>

</math><img src="./890f2f9369b6f727ae5f31082164cd8dd718344c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:9.158ex; height:2.843ex;" alt="{\textstyle (R,A,Y)}" loading="lazy"> satisfy sufficiency if the sensitive characteristics <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy"> are <a href="Independence_(probability_theory)" title="Independence (probability theory)">statistically independent</a> of the target value <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> given the prediction <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle R}</annotation>

</semantics>

</math><img src="./197e66194eb64577670e2a100026bff6fb15d236.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\textstyle R}" loading="lazy">, and we write

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Y\bot A\ |\ R.}</annotation>

</semantics>

</math>

We can also express this notion with the following formula:

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(Y=q\ |\ R=r,A=a)=P(Y=q\ |\ R=r,A=b)\quad \forall q\in Y\quad r\in R\quad \forall a,b\in A}</annotation>

</semantics>

</math>

This means that the <a href="Probability_theory" title="Probability theory">probability</a> of actually being in each of the groups is equal for two individuals with different sensitive characteristics given that they were predicted to belong to the same group.

<div class="mw-heading mw-heading3"><h3 id="Relationships_between_definitions">Relationships between definitions</h3></div>

Finally, we sum up some of the main results that relate the three definitions given above:

<ul><li>Assuming <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> is binary, if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> are not <a href="Independence_(probability_theory)" title="Independence (probability theory)">statistically independent</a>, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle R}</annotation>

</semantics>

</math><img src="./197e66194eb64577670e2a100026bff6fb15d236.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\textstyle R}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> are not <a href="Independence_(probability_theory)" title="Independence (probability theory)">statistically independent</a> either, then independence and separation cannot both hold except for rhetorical cases.</li>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle (R,A,Y)}</annotation>

</semantics>

</math><img src="./890f2f9369b6f727ae5f31082164cd8dd718344c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:9.158ex; height:2.843ex;" alt="{\textstyle (R,A,Y)}" loading="lazy"> as a <a href="Joint_distribution" class="mw-redirect" title="Joint distribution">joint distribution</a> has positive <a href="Probability_theory" title="Probability theory">probability</a> for all its possible values and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> are not <a href="Independence_(probability_theory)" title="Independence (probability theory)">statistically independent</a>, then separation and sufficiency cannot both hold except for rhetorical cases.</li></ul>

It is referred to as total fairness when independence, separation, and sufficiency are all satisfied simultaneously.<a href="#cite_note-34">[34]</a> However, total fairness is not possible to achieve except in specific rhetorical cases.<a href="#cite_note-Räz_129–137-35">[35]</a>

<div class="mw-heading mw-heading3"><h3 id="Mathematical_formulation_of_group_fairness_definitions">Mathematical formulation of group fairness definitions</h3></div>

<div class="mw-heading mw-heading4"><h4 id="Preliminary_definitions">Preliminary definitions</h4></div>

<div role="note" class="hatnote navigation-not-searchable">Main article: <a href="Confusion_matrix" title="Confusion matrix">Confusion matrix</a></div>

Most statistical measures of fairness rely on different metrics, so we will start by defining them. When working with a <a href="Binary_numeral_system" class="mw-redirect" title="Binary numeral system">binary</a> classifier, both the predicted and the actual classes can take two values: positive and negative. Now let us start explaining the different possible relations between predicted and actual outcome:<a href="#cite_note-metrics_paper-36">[36]</a>

<ul><li>True positive (TP): The case where both the predicted and the actual outcome are in a positive class.</li>

<li>True negative (TN): The case where both the predicted outcome and the actual outcome are assigned to the negative class.</li>

<li>False positive (FP): A case predicted to befall into a positive class assigned in the actual outcome is to the negative one.</li>

<li>False negative (FN): A case predicted to be in the negative class with an actual outcome is in the positive one.</li></ul>

These relations can be easily represented with a <a href="Confusion_matrix" title="Confusion matrix">confusion matrix</a>, a table that describes the accuracy of a classification model. In this matrix, columns and rows represent instances of the predicted and the actual cases, respectively.

By using these relations, we can define multiple metrics which can be later used to measure the fairness of an algorithm:

<ul><li>Positive predicted value (PPV): the fraction of positive cases which were correctly predicted out of all the positive predictions. It is usually referred to as <a href="Accuracy_and_precision" title="Accuracy and precision">precision</a>, and represents the <a href="Probability_theory" title="Probability theory">probability</a> of a correct positive prediction. It is given by the following formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle PPV=P(actual=+\ |\ prediction=+)={\frac {TP}{TP+FP}}}">

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle PPV=P(actual=+\ |\ prediction=+)={\frac {TP}{TP+FP}}}</annotation>

</semantics>

</math></li>

<li>False discovery rate (FDR): the fraction of positive predictions which were actually negative out of all the positive predictions. It represents the <a href="Probability_theory" title="Probability theory">probability</a> of an erroneous positive prediction, and it is given by the following formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle FDR=P(actual=-\ |\ prediction=+)={\frac {FP}{TP+FP}}}">

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle FDR=P(actual=-\ |\ prediction=+)={\frac {FP}{TP+FP}}}</annotation>

</semantics>

</math></li>

<li>Negative predicted value (NPV): the fraction of negative cases which were correctly predicted out of all the negative predictions. It represents the <a href="Probability_theory" title="Probability theory">probability</a> of a correct negative prediction, and it is given by the following formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle NPV=P(actual=-\ |\ prediction=-)={\frac {TN}{TN+FN}}}">

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle NPV=P(actual=-\ |\ prediction=-)={\frac {TN}{TN+FN}}}</annotation>

</semantics>

</math></li>

<li>False omission rate (FOR): the fraction of negative predictions which were actually positive out of all the negative predictions. It represents the <a href="Probability_theory" title="Probability theory">probability</a> of an erroneous negative prediction, and it is given by the following formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle FOR=P(actual=+\ |\ prediction=-)={\frac {FN}{TN+FN}}}">

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle FOR=P(actual=+\ |\ prediction=-)={\frac {FN}{TN+FN}}}</annotation>

</semantics>

</math></li>

<li>True positive rate (TPR): the fraction of positive cases which were correctly predicted out of all the positive cases. It is usually referred to as sensitivity or recall, and it represents the <a href="Probability_theory" title="Probability theory">probability</a> of the positive subjects to be classified correctly as such. It is given by the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle TPR=P(prediction=+\ |\ actual=+)={\frac {TP}{TP+FN}}}">

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle TPR=P(prediction=+\ |\ actual=+)={\frac {TP}{TP+FN}}}</annotation>

</semantics>

</math></li>

<li>False negative rate (FNR): the fraction of positive cases which were incorrectly predicted to be negative out of all the positive cases. It represents the <a href="Probability_theory" title="Probability theory">probability</a> of the positive subjects to be classified incorrectly as negative ones, and it is given by the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle FNR=P(prediction=-\ |\ actual=+)={\frac {FN}{TP+FN}}}">

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle FNR=P(prediction=-\ |\ actual=+)={\frac {FN}{TP+FN}}}</annotation>

</semantics>

</math></li>

<li>True negative rate (TNR): the fraction of negative cases which were correctly predicted out of all the negative cases. It represents the <a href="Probability_theory" title="Probability theory">probability</a> of the negative subjects to be classified correctly as such, and it is given by the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle TNR=P(prediction=-\ |\ actual=-)={\frac {TN}{TN+FP}}}">

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle TNR=P(prediction=-\ |\ actual=-)={\frac {TN}{TN+FP}}}</annotation>

</semantics>

</math></li>

<li>False positive rate (FPR): the fraction of negative cases which were incorrectly predicted to be positive out of all the negative cases. It represents the <a href="Probability_theory" title="Probability theory">probability</a> of the negative subjects to be classified incorrectly as positive ones, and it is given by the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle FPR=P(prediction=+\ |\ actual=-)={\frac {FP}{TN+FP}}}">

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle FPR=P(prediction=+\ |\ actual=-)={\frac {FP}{TN+FP}}}</annotation>

</semantics>

</math></li></ul>

The following criteria can be understood as measures of the three general definitions given at the beginning of this section, namely Independence, Separation and Sufficiency. In the table<a href="#cite_note-Barocas-32">[32]</a> to the right, we can see the relationships between them.

To define these measures specifically, we will divide them into three big groups as done in Verma et al.:<a href="#cite_note-metrics_paper-36">[36]</a> definitions based on a predicted outcome, on predicted and actual outcomes, and definitions based on predicted probabilities and the actual outcome.

We will be working with a binary classifier and the following notation: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy"> refers to the score given by the classifier, which is the probability of a certain subject to be in the positive or the negative class. <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle R}</annotation>

</semantics>

</math><img src="./197e66194eb64577670e2a100026bff6fb15d236.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\textstyle R}" loading="lazy"> represents the final classification predicted by the algorithm, and its value is usually derived from <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy">, for example will be positive when <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy"> is above a certain threshold. <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> represents the actual outcome, that is, the real classification of the individual and, finally, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy"> denotes the sensitive attributes of the subjects.

<div class="mw-heading mw-heading4"><h4 id="Definitions_based_on_predicted_outcome">Definitions based on predicted outcome</h4></div>

The definitions in this section focus on a predicted outcome <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle R}</annotation>

</semantics>

</math><img src="./197e66194eb64577670e2a100026bff6fb15d236.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\textstyle R}" loading="lazy"> for various <a href="Probability_distribution" title="Probability distribution">distributions</a> of subjects. They are the simplest and most intuitive notions of fairness.

<ul><li>Demographic parity, also referred to as statistical parity, acceptance rate parity and benchmarking. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal probability of being assigned to the positive predicted class. This is, if the following formula is satisfied:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(R=+\ |\ A=a)=P(R=+\ |\ A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=+\ |\ A=a)=P(R=+\ |\ A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></li>

<li>Conditional statistical parity. Basically consists in the definition above, but restricted only to a <a href="Subset" title="Subset">subset</a> of the instances. In mathematical notation this would be:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(R=+\ |\ L=l,A=a)=P(R=+\ |\ L=l,A=b)\quad \forall a,b\in A\quad \forall l\in L}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=+\ |\ L=l,A=a)=P(R=+\ |\ L=l,A=b)\quad \forall a,b\in A\quad \forall l\in L}</annotation>

</semantics>

</math></li></ul>

<div class="mw-heading mw-heading4"><h4 id="Definitions_based_on_predicted_and_actual_outcomes">Definitions based on predicted and actual outcomes</h4></div>

These definitions not only considers the predicted outcome <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle R}</annotation>

</semantics>

</math><img src="./197e66194eb64577670e2a100026bff6fb15d236.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\textstyle R}" loading="lazy"> but also compare it to the actual outcome <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy">.

<ul><li>Predictive parity, also referred to as outcome test. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal PPV. This is, if the following formula is satisfied:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(Y=+\ |\ R=+,A=a)=P(Y=+\ |\ R=+,A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(Y=+\ |\ R=+,A=a)=P(Y=+\ |\ R=+,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></li></ul>

<dl><dd>Mathematically, if a classifier has equal PPV for both groups, it will also have equal FDR, satisfying the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(Y=-\ |\ R=+,A=a)=P(Y=-\ |\ R=+,A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(Y=-\ |\ R=+,A=a)=P(Y=-\ |\ R=+,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></dd></dl>

<ul><li>False positive error rate balance, also referred to as predictive equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal FPR. This is, if the following formula is satisfied:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(R=+\ |\ Y=-,A=a)=P(R=+\ |\ Y=-,A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=+\ |\ Y=-,A=a)=P(R=+\ |\ Y=-,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></li></ul>

<dl><dd>Mathematically, if a classifier has equal FPR for both groups, it will also have equal TNR, satisfying the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(R=-\ |\ Y=-,A=a)=P(R=-\ |\ Y=-,A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=-\ |\ Y=-,A=a)=P(R=-\ |\ Y=-,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></dd></dl>

<ul><li>False negative error rate balance, also referred to as equal opportunity. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal FNR. This is, if the following formula is satisfied:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(R=-\ |\ Y=+,A=a)=P(R=-\ |\ Y=+,A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=-\ |\ Y=+,A=a)=P(R=-\ |\ Y=+,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></li></ul>

<dl><dd>Mathematically, if a classifier has equal FNR for both groups, it will also have equal TPR, satisfying the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(R=+\ |\ Y=+,A=a)=P(R=+\ |\ Y=+,A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=+\ |\ Y=+,A=a)=P(R=+\ |\ Y=+,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></dd></dl>

<ul><li><a href="Equalized_odds" title="Equalized odds">Equalized odds</a>, also referred to as conditional procedure accuracy equality and disparate mistreatment. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal TPR and equal FPR, satisfying the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(R=+\ |\ Y=y,A=a)=P(R=+\ |\ Y=y,A=b)\quad y\in \{+,-\}\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=+\ |\ Y=y,A=a)=P(R=+\ |\ Y=y,A=b)\quad y\in \{+,-\}\quad \forall a,b\in A}</annotation>

</semantics>

</math></li>

<li>Conditional use accuracy equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal PPV and equal NPV, satisfying the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(Y=y\ |\ R=y,A=a)=P(Y=y\ |\ R=y,A=b)\quad y\in \{+,-\}\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(Y=y\ |\ R=y,A=a)=P(Y=y\ |\ R=y,A=b)\quad y\in \{+,-\}\quad \forall a,b\in A}</annotation>

</semantics>

</math></li>

<li>Overall accuracy equality. A classifier satisfies this definition if the subject in the protected and unprotected groups have equal prediction accuracy, that is, the probability of a subject from one class to be assigned to it. This is, if it satisfies the following formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(R=Y\ |\ A=a)=P(R=Y\ |\ A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=Y\ |\ A=a)=P(R=Y\ |\ A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></li>

<li>Treatment equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have an equal ratio of FN and FP, satisfying the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\frac {FN_{A=a}}{FP_{A=a}}}={\frac {FN_{A=b}}{FP_{A=b}}}}">

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {FN_{A=a}}{FP_{A=a}}}={\frac {FN_{A=b}}{FP_{A=b}}}}</annotation>

</semantics>

</math></li></ul>

<div class="mw-heading mw-heading4"><h4 id="Definitions_based_on_predicted_probabilities_and_actual_outcome">Definitions based on predicted probabilities and actual outcome</h4></div>

These definitions are based in the actual outcome <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> and the predicted probability score <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy">.

<ul><li>Test-fairness, also known as calibration or matching conditional frequencies. A classifier satisfies this definition if individuals with the same predicted probability score <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy"> have the same probability of being classified in the positive class when they belong to either the protected or the unprotected group:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(Y=+\ |\ S=s,A=a)=P(Y=+\ |\ S=s,A=b)\quad \forall s\in S\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(Y=+\ |\ S=s,A=a)=P(Y=+\ |\ S=s,A=b)\quad \forall s\in S\quad \forall a,b\in A}</annotation>

</semantics>

</math></li>

<li>Well-calibration is an extension of the previous definition. It states that when individuals inside or outside the protected group have the same predicted probability score <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy"> they must have the same probability of being classified in the positive class, and this probability must be equal to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy">:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(Y=+\ |\ S=s,A=a)=P(Y=+\ |\ S=s,A=b)=s\quad \forall s\in S\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(Y=+\ |\ S=s,A=a)=P(Y=+\ |\ S=s,A=b)=s\quad \forall s\in S\quad \forall a,b\in A}</annotation>

</semantics>

</math></li>

<li>Balance for positive class. A classifier satisfies this definition if the subjects constituting the positive class from both protected and unprotected groups have equal average predicted probability score <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy">. This means that the expected value of probability score for the protected and unprotected groups with positive actual outcome <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> is the same, satisfying the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle E(S\ |\ Y=+,A=a)=E(S\ |\ Y=+,A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle E(S\ |\ Y=+,A=a)=E(S\ |\ Y=+,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></li>

<li>Balance for negative class. A classifier satisfies this definition if the subjects constituting the negative class from both protected and unprotected groups have equal average predicted probability score <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle S}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle S}</annotation>

</semantics>

</math><img src="./e10a3c52d186162ec8910ebc0288ce982aef842f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.499ex; height:2.176ex;" alt="{\textstyle S}" loading="lazy">. This means that the expected value of probability score for the protected and unprotected groups with negative actual outcome <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> is the same, satisfying the formula:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle E(S\ |\ Y=-,A=a)=E(S\ |\ Y=-,A=b)\quad \forall a,b\in A}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle E(S\ |\ Y=-,A=a)=E(S\ |\ Y=-,A=b)\quad \forall a,b\in A}</annotation>

</semantics>

</math></li></ul>

<div class="mw-heading mw-heading3"><h3 id="Equal_confusion_fairness">Equal confusion fairness</h3></div>

With respect to <a href="Confusion_matrix" title="Confusion matrix">confusion matrices</a>, independence, separation, and sufficiency require the respective quantities listed below to not have statistically significant difference across sensitive characteristics.<a href="#cite_note-Räz_129–137-35">[35]</a>

<ul><li>Independence: (TP + FP) / (TP + FP + FN + TN) (i.e., <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P({\hat {Y}}=1)}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P({\hat {Y}}=1)}</annotation>

</semantics>

</math><img src="./af8c5ba3e6a0eaa3f0a6067e26fc49afcb5b0583.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:9.589ex; height:3.343ex;" alt="{\displaystyle P({\hat {Y}}=1)}" loading="lazy">).</li>

<li>Separation: TN / (TN + FP) and TP / (TP + FN) (i.e., specificity <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P({\hat {Y}}=0\mid Y=0)}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P({\hat {Y}}=0\mid Y=0)}</annotation>

</semantics>

</math><img src="./a89e6ba37521eda53b3cc898b836c07739f20663.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:17.56ex; height:3.343ex;" alt="{\displaystyle P({\hat {Y}}=0\mid Y=0)}" loading="lazy"> and recall <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P({\hat {Y}}=1\mid Y=1)}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P({\hat {Y}}=1\mid Y=1)}</annotation>

</semantics>

</math><img src="./1619a33bd4a8fe34623af230c5ea2165233558ec.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:17.56ex; height:3.343ex;" alt="{\displaystyle P({\hat {Y}}=1\mid Y=1)}" loading="lazy">).</li>

<li>Sufficiency: TP / (TP + FP) and TN / (TN + FN) (i.e., precision <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(Y=1\mid {\hat {Y}}=1)}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(Y=1\mid {\hat {Y}}=1)}</annotation>

</semantics>

</math><img src="./a588ef4324fc48814717c59e2645e9617216dd2d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:17.56ex; height:3.343ex;" alt="{\displaystyle P(Y=1\mid {\hat {Y}}=1)}" loading="lazy"> and negative predictive value <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P(Y=0\mid {\hat {Y}}=0)}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(Y=0\mid {\hat {Y}}=0)}</annotation>

</semantics>

</math><img src="./fb2c873466c0e607ded4236aad1c4c427dab6e1f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:17.56ex; height:3.343ex;" alt="{\displaystyle P(Y=0\mid {\hat {Y}}=0)}" loading="lazy">).</li></ul>

The notion of equal confusion fairness<a href="#cite_note-37">[37]</a> requires the confusion matrix of a given decision system to have the same distribution when computed stratified over all sensitive characteristics.

<div class="mw-heading mw-heading3"><h3 id="Social_welfare_function">Social welfare function</h3></div>

Some scholars have proposed defining algorithmic fairness in terms of a <a href="Social_welfare_function" title="Social welfare function">social welfare function</a>. They argue that using a social welfare function enables an algorithm designer to consider fairness and predictive accuracy in terms of their benefits to the people affected by the algorithm. It also allows the designer to <a href="Trade_off" class="mw-redirect" title="Trade off">trade off</a> efficiency and equity in a principled way.<a href="#cite_note-chen-hooker-2021-38">[38]</a> <a href="Sendhil_Mullainathan" title="Sendhil Mullainathan">Sendhil Mullainathan</a> has stated that algorithm designers should use social welfare functions to recognize absolute gains for disadvantaged groups. For example, a study found that using a decision-making algorithm in <a href="Pretrial_detention" class="mw-redirect" title="Pretrial detention">pretrial detention</a> rather than pure human judgment reduced the detention rates for Blacks, Hispanics, and racial minorities overall, even while keeping the crime rate constant.<a href="#cite_note-mullainathan-ec-2018-39">[39]</a>

<div class="mw-heading mw-heading2"><h2 id="Individual_fairness_criteria">Individual fairness criteria</h2></div>

An important distinction among fairness definitions is the one between group and individual notions.<a href="#cite_note-mitchell2021-40">[40]</a><a href="#cite_note-castelnovo2022-41">[41]</a><a href="#cite_note-metrics_paper-36">[36]</a><a href="#cite_note-mehrabi2021-42">[42]</a> Roughly speaking, while group fairness criteria compare quantities at a group level, typically identified by sensitive attributes (e.g. gender, ethnicity, age, etc.), individual criteria compare individuals. In words, individual fairness follow the principle that "similar individuals should receive similar treatments".

There is a very intuitive approach to fairness, which usually goes under the name of fairness through unawareness (FTU), or blindness, that prescribes not to explicitly employ sensitive features when making (automated) decisions. This is effectively a notion of individual fairness, since two individuals differing only for the value of their sensitive attributes would receive the same outcome.

However, in general, FTU is subject to several drawbacks, the main being that it does not take into account possible correlations between sensitive attributes and non-sensitive attributes employed in the decision-making process. For example, an agent with the (malignant) intention to discriminate on the basis of gender could introduce in the model a proxy variable for gender (i.e. a variable highly correlated with gender) and effectively using gender information while at the same time being compliant to the FTU prescription.

The problem of what variables correlated to sensitive ones are fairly employable by a model in the decision-making process is a crucial one, and is relevant for <a href="#Group_fairness_criteria">group concepts</a> as well: independence metrics require a complete removal of sensitive information, while separation-based metrics allow for correlation, but only as far as the labeled target variable "justify" them.

The most general concept of individual fairness was introduced in the pioneer work by <a href="Cynthia_Dwork" title="Cynthia Dwork">Cynthia Dwork</a> and collaborators in 2012<a href="#cite_note-43">[43]</a> and can be thought of as a mathematical translation of the principle that the decision map taking features as input should be built such that it is able to "map similar individuals similarly", that is expressed as a <a href="Lipschitz_continuity" title="Lipschitz continuity">Lipschitz condition</a> on the model map. They call this approach fairness through awareness (FTA), precisely as counterpoint to FTU, since they underline the importance of choosing the appropriate target-related distance metric to assess which individuals are similar in specific situations. Again, this problem is very related to the point raised above about what variables can be seen as "legitimate" in particular contexts.

<div class="mw-heading mw-heading2"><h2 id="Causality-based_metrics">Causality-based metrics</h2></div>

Causal fairness measures the frequency with which two nearly identical users or applications who differ only in a set of characteristics with respect to which resource allocation must be fair receive identical treatment.<a href="#cite_note-causal-44">[44]</a>

An entire branch of the academic research on fairness metrics is devoted to leverage causal models to assess bias in <a href="Machine_learning" title="Machine learning">machine learning</a> models. This approach is usually justified by the fact that the same observational distribution of data may hide different causal relationships among the variables at play, possibly with different interpretations of whether the outcome are affected by some form of bias or not.<a href="#cite_note-Barocas-32">[32]</a>

Kusner et al.<a href="#cite_note-cff-45">[45]</a> propose to employ <a href="Causal_model#Counterfactuals" title="Causal model">counterfactuals</a>, and define a decision-making process counterfactually fair if, for any individual, the outcome does not change in the counterfactual scenario where the sensitive attributes are changed. The mathematical formulation reads:

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R_{A\leftarrow a}=1\mid A=a,X=x)=P(R_{A\leftarrow b}=1\mid A=a,X=x),\quad \forall a,b;}</annotation>

</semantics>

</math><img src="./37e32da6b853d0340e152cef64a4727b28fff45b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:69.507ex; height:2.843ex;" alt="{\displaystyle P(R_{A\leftarrow a}=1\mid A=a,X=x)=P(R_{A\leftarrow b}=1\mid A=a,X=x),\quad \forall a,b;}" loading="lazy">

that is: taken a random individual with sensitive attribute <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A=a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A=a}</annotation>

</semantics>

</math><img src="./6509e2e3a7f69da0109d3df052c95bb1c12f1e95.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.071ex; height:2.176ex;" alt="{\displaystyle A=a}" loading="lazy"> and other features <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X=x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X=x}</annotation>

</semantics>

</math><img src="./0661396d873679039ffe8e908a39f02402d4912d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.408ex; height:2.176ex;" alt="{\displaystyle X=x}" loading="lazy"> and the same individual if she had <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A=b}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A=b}</annotation>

</semantics>

</math><img src="./179ea8f8ceb42d1c488234b1729273b6d878cd83.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.839ex; height:2.176ex;" alt="{\displaystyle A=b}" loading="lazy">, they should have same chance of being accepted.

The symbol <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\hat {R}}_{A\leftarrow a}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\hat {R}}_{A\leftarrow a}}</annotation>

</semantics>

</math><img src="./7c88be5bd6279378c3f1fe3b82d5d85ce6be58d8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:5.742ex; height:3.176ex;" alt="{\displaystyle {\hat {R}}_{A\leftarrow a}}" loading="lazy"> represents the counterfactual random variable <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle R}</annotation>

</semantics>

</math><img src="./4b0bfb3769bf24d80e15374dc37b0441e2616e33.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\displaystyle R}" loading="lazy"> in the scenario where the sensitive attribute <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy"> is fixed to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A=a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A=a}</annotation>

</semantics>

</math><img src="./6509e2e3a7f69da0109d3df052c95bb1c12f1e95.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.071ex; height:2.176ex;" alt="{\displaystyle A=a}" loading="lazy">. The conditioning on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A=a,X=x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A=a,X=x}</annotation>

</semantics>

</math><img src="./c06895c2088c6dad6a1dbbc19fdf6f9c7b5f62a0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:13.513ex; height:2.509ex;" alt="{\displaystyle A=a,X=x}" loading="lazy"> means that this requirement is at the individual level, in that we are conditioning on all the variables identifying a single observation.

Machine learning models are often trained upon data where the outcome depended on the decision made at that time.<a href="#cite_note-46">[46]</a> For example, if a machine learning model has to determine whether an inmate will recidivate and will determine whether the inmate should be released early, the outcome could be dependent on whether the inmate was released early or not. Mishler et al.<a href="#cite_note-47">[47]</a> propose a formula for counterfactual equalized odds:

<msup>

</mrow>

</msup>

<msup>

</mrow>

</msup>

<msup>

</mrow>

</msup>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P(R=1\mid Y^{0}=0,A=a)=P(R=1\mid Y^{0}=0,A=b)\wedge P(R=0\mid Y^{1}=1,A=a)=P(R=0\mid Y^{1}=1,A=b),\quad \forall a,b;}</annotation>

</semantics>

</math><img src="./408aea5037eb4f5e52dcfe9937daad7213acd431.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:120.222ex; height:3.176ex;" alt="{\displaystyle P(R=1\mid Y^{0}=0,A=a)=P(R=1\mid Y^{0}=0,A=b)\wedge P(R=0\mid Y^{1}=1,A=a)=P(R=0\mid Y^{1}=1,A=b),\quad \forall a,b;}" loading="lazy">

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle R}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle R}</annotation>

</semantics>

</math><img src="./4b0bfb3769bf24d80e15374dc37b0441e2616e33.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.764ex; height:2.176ex;" alt="{\displaystyle R}" loading="lazy"> is a random variable, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Y^{x}}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Y^{x}}</annotation>

</semantics>

</math><img src="./5b9a952b8d3784b700d260e8de61ad601485c8e8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:3.073ex; height:2.176ex;" alt="{\displaystyle Y^{x}}" loading="lazy"> denotes the outcome given that the decision <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x}</annotation>

</semantics>

</math><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy"> was taken, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy"> is a sensitive feature.

Plecko and Bareinboim<a href="#cite_note-Plecko_Bareinboim_p.-48">[48]</a> propose a unified framework to deal with causal analysis of fairness. They suggest the use of a Standard Fairness Model, consisting of a causal graph with 4 types of variables:

<ul><li>sensitive attributes (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy">),</li>

<li>target variable (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Y}</annotation>

</semantics>

</math><img src="./961d67d6b454b4df2301ac571808a3538b3a6d3f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\displaystyle Y}" loading="lazy">),</li>

<li>mediators (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W}</annotation>

</semantics>

</math><img src="./54a9c4c547f4d6111f81946cad242b18298d70b7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\displaystyle W}" loading="lazy">) between <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Y}</annotation>

</semantics>

</math><img src="./961d67d6b454b4df2301ac571808a3538b3a6d3f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\displaystyle Y}" loading="lazy">, representing possible indirect effects of sensitive attributes on the outcome,</li>

<li>variables possibly sharing a common cause with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A}</annotation>

</semantics>

</math><img src="./7daff47fa58cdfd29dc333def748ff5fa4c923e3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\displaystyle A}" loading="lazy"> (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z}</annotation>

</semantics>

</math><img src="./1cc6b75e09a8aa3f04d8584b11db534f88fb56bd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.68ex; height:2.176ex;" alt="{\displaystyle Z}" loading="lazy">), representing possible spurious (i.e., non causal) effects of the sensitive attributes on the outcome.</li></ul>

Within this framework, Plecko and Bareinboim<a href="#cite_note-Plecko_Bareinboim_p.-48">[48]</a> are therefore able to classify the possible effects that sensitive attributes may have on the outcome.

Moreover, the granularity at which these effects are measured—namely, the conditioning variables used to average the effect—is directly connected to the "individual vs. group" aspect of fairness assessment.

<div class="mw-heading mw-heading2"><h2 id="Bias_mitigation_strategies">Bias mitigation strategies</h2></div>

Fairness can be applied to machine learning algorithms in three different ways: <a href="Data_preprocessing" title="Data preprocessing">data preprocessing</a>, <a href="Mathematical_optimization" title="Mathematical optimization">optimization</a> during software training, or post-processing results of the algorithm.

<div class="mw-heading mw-heading3"><h3 id="Preprocessing">Preprocessing</h3></div>

Usually, the classifier is not the only problem; the <a href="Dataset" class="mw-redirect" title="Dataset">dataset</a> is also biased. The discrimination of a dataset <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle D}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle D}</annotation>

</semantics>

</math><img src="./4e5200f518cb5afe304ec42ffdd4f6c63c702f77.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.924ex; height:2.176ex;" alt="{\textstyle D}" loading="lazy"> with respect to the group <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A=a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A=a}</annotation>

</semantics>

</math><img src="./ee403632afa0201f2ca58ef03fd95032b60dbbed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.071ex; height:2.176ex;" alt="{\textstyle A=a}" loading="lazy"> can be defined as follows:

<msub>

</mrow>

</msub>

<mfrac>

<mrow>

</mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

</mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle disc_{A=a}(D)={\frac {|\{X\in D|X(A)\neq a,X(Y)=+\}|}{|\{X\in D|X(A)\neq a\}|}}-{\frac {|\{X\in D|X(A)=a,X(Y)=+\}|}{|\{X\in D|X(A)=a\}|}}}</annotation>

</semantics>

</math>

That is, an approximation to the difference between the probabilities of belonging in the positive class given that the subject has a protected characteristic different from <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle a}</annotation>

</semantics>

</math><img src="./7a503f107a7c104e40e484cee9e1f5993d28ffd8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.23ex; height:1.676ex;" alt="{\textstyle a}" loading="lazy"> and equal to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle a}</annotation>

</semantics>

</math><img src="./7a503f107a7c104e40e484cee9e1f5993d28ffd8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.23ex; height:1.676ex;" alt="{\textstyle a}" loading="lazy">.

Algorithms correcting bias at preprocessing remove information about dataset variables which might result in unfair decisions, while trying to alter as little as possible. This is not as simple as just removing the sensitive variable, because other attributes can be correlated to the protected one.

A way to do this is to map each individual in the initial dataset to an intermediate representation in which it is impossible to identify whether it belongs to a particular protected group while maintaining as much information as possible. Then, the new representation of the data is adjusted to get the maximum accuracy in the algorithm.

This way, individuals are mapped into a new multivariable representation where the probability of any member of a protected group to be mapped to a certain value in the new representation is the same as the probability of an individual which doesn't belong to the protected group. Then, this representation is used to obtain the prediction for the individual, instead of the initial data. As the intermediate representation is constructed giving the same probability to individuals inside or outside the protected group, this attribute is hidden to the classifier.

An example is explained in Zemel et al.<a href="#cite_note-zemel-49">[49]</a> where a <a href="Multinomial_distribution" title="Multinomial distribution">multinomial random variable</a> is used as an intermediate representation. In the process, the system is encouraged to preserve all information except that which can lead to biased decisions, and to obtain a prediction as accurate as possible.

On the one hand, this procedure has the advantage that the preprocessed data can be used for any machine learning task. Furthermore, the classifier does not need to be modified, as the correction is applied to the <a href="Data_set" title="Data set">dataset</a> before processing. On the other hand, the other methods obtain better results in accuracy and fairness.

<div class="mw-heading mw-heading4"><h4 id="Reweighing">Reweighing</h4></div>

Reweighing is an example of a preprocessing algorithm. The idea is to assign a weight to each dataset point such that the weighted discrimination is 0 with respect to the designated group.<a href="#cite_note-reweighing-50">[50]</a>

If the dataset <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle D}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle D}</annotation>

</semantics>

</math><img src="./4e5200f518cb5afe304ec42ffdd4f6c63c702f77.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.924ex; height:2.176ex;" alt="{\textstyle D}" loading="lazy"> was unbiased the sensitive variable <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy"> and the target variable <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy"> would be <a href="Independence_(probability_theory)" title="Independence (probability theory)">statistically independent</a> and the probability of the <a href="Joint_probability_distribution" title="Joint probability distribution">joint distribution</a> would be the product of the probabilities as follows:

<msub>

</mrow>

</msub>

<mfrac>

<mrow>

</mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

</mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P_{exp}(A=a\wedge Y=+)=P(A=a)\times P(Y=+)={\frac {|\{X\in D|X(A)=a\}|}{|D|}}\times {\frac {|\{X\in D|X(Y)=+\}|}{|D|}}}</annotation>

</semantics>

</math>

In reality, however, the dataset is not unbiased and the variables are not <a href="Independence_(probability_theory)" title="Independence (probability theory)">statistically independent</a> so the observed probability is:

<msub>

</mrow>

</msub>

<mfrac>

<mrow>

</mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P_{obs}(A=a\wedge Y=+)={\frac {|\{X\in D|X(A)=a\wedge X(Y)=+\}|}{|D|}}}</annotation>

</semantics>

</math>

To compensate for the bias, the software adds a <a href="Weight_function" title="Weight function">weight</a>, lower for favored objects and higher for unfavored objects. For each <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle X\in D}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X\in D}</annotation>

</semantics>

</math><img src="./41650474e2ac80e5caf1cbe679a61d7a4b4bc0e5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.745ex; height:2.176ex;" alt="{\textstyle X\in D}" loading="lazy"> we get:

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W(X)={\frac {P_{exp}(A=X(A)\wedge Y=X(Y))}{P_{obs}(A=X(A)\wedge Y=X(Y))}}}</annotation>

</semantics>

</math>

When we have for each <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X}</annotation>

</semantics>

</math><img src="./8d80c41192705e1a6c6de1d65e16d7f70fbac391.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\textstyle X}" loading="lazy"> a weight associated <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle W(X)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle W(X)}</annotation>

</semantics>

</math><img src="./be800caaad2bd519597cbc6c85a7c23f87d6ae49.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:6.225ex; height:2.843ex;" alt="{\textstyle W(X)}" loading="lazy"> we compute the weighted discrimination with respect to group <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A=a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A=a}</annotation>

</semantics>

</math><img src="./ee403632afa0201f2ca58ef03fd95032b60dbbed.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.071ex; height:2.176ex;" alt="{\textstyle A=a}" loading="lazy"> as follows:

<msub>

</mrow>

</msub>

<mfrac>

<mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle disc_{A=a}(D)={\frac {\sum W(X)X\in \{X\in D|X(A)\neq a,X(Y)=+\}}{\sum W(X)X\in \{X\in D|X(A)\neq a\}}}-{\frac {\sum W(X)X\in \{X\in D|X(A)=a,X(Y)=+\}}{\sum W(X)X\in \{X\in D|X(A)=a\}}}}</annotation>

</semantics>

</math>

It can be shown that after reweighting this weighted discrimination is 0.

<div class="mw-heading mw-heading3"><h3 id="Inprocessing">Inprocessing</h3></div>

Another approach is to correct the <a href="Bias" title="Bias">bias</a> at training time. This can be done by adding constraints to the optimization objective of the algorithm.<a href="#cite_note-zafar-51">[51]</a> These constraints force the algorithm to improve fairness, by keeping the same rates of certain measures for the protected group and the rest of individuals. For example, we can add to the objective of the <a href="Algorithm" title="Algorithm">algorithm</a> the condition that the false positive rate is the same for individuals in the protected group and the ones outside the protected group.

The main measures used in this approach are false positive rate, false negative rate, and overall misclassification rate. It is possible to add just one or several of these constraints to the objective of the algorithm. Note that the equality of false negative rates implies the equality of true positive rates so this implies the equality of opportunity. After adding the restrictions to the problem it may turn intractable, so a relaxation on them may be needed.

<div class="mw-heading mw-heading4"><h4 id="Adversarial_debiasing">Adversarial debiasing</h4></div>

We train two <a href="Statistical_classification" title="Statistical classification">classifiers</a> at the same time through some gradient-based method (f.e.: <a href="Gradient_descent" title="Gradient descent">gradient descent</a>). The first one, the predictor tries to accomplish the task of predicting <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle Y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle Y}</annotation>

</semantics>

</math><img src="./6222becc4f0c5effa012e5335b170575fdbbaad3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.009ex;" alt="{\textstyle Y}" loading="lazy">, the target variable, given <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X}</annotation>

</semantics>

</math><img src="./8d80c41192705e1a6c6de1d65e16d7f70fbac391.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\textstyle X}" loading="lazy">, the input, by modifying its weights <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle W}</annotation>

</semantics>

</math><img src="./e95737ee2530885a10b104e9cd5331077e1c88d5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\textstyle W}" loading="lazy"> to minimize some <a href="Loss_function" title="Loss function">loss function</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle L_{P}({\hat {y}},y)}">

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle L_{P}({\hat {y}},y)}</annotation>

</semantics>

</math><img src="./d2b5d649b4fd45b4fae2f55635e3ab14fba0eba9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.35ex; height:2.843ex;" alt="{\textstyle L_{P}({\hat {y}},y)}" loading="lazy">. The second one, the adversary tries to accomplish the task of predicting <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle A}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle A}</annotation>

</semantics>

</math><img src="./a118c6ad00742b3f5dccd2f0e74b5e369df6fd31.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.743ex; height:2.176ex;" alt="{\textstyle A}" loading="lazy">, the sensitive variable, given <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle {\hat {Y}}}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle {\hat {Y}}}</annotation>

</semantics>

</math><img src="./d62b45891b318ecf6599faa7421e504490df672a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.676ex;" alt="{\textstyle {\hat {Y}}}" loading="lazy"> by modifying its weights <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle U}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle U}</annotation>

</semantics>

</math><img src="./087a7d8a39fe35012cbf2f561879f9e975cb4555.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.783ex; height:2.176ex;" alt="{\textstyle U}" loading="lazy"> to minimize some loss function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle L_{A}({\hat {a}},a)}">

<msub>

</mrow>

</msub>

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle L_{A}({\hat {a}},a)}</annotation>

</semantics>

</math><img src="./9875946fe48609c203c201ed9d96cae8a0249fd8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.351ex; height:2.843ex;" alt="{\textstyle L_{A}({\hat {a}},a)}" loading="lazy">.<a href="#cite_note-adversarial1-52">[52]</a>

An important point here is that, to propagate correctly, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle {\hat {Y}}}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle {\hat {Y}}}</annotation>

</semantics>

</math><img src="./d62b45891b318ecf6599faa7421e504490df672a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.676ex;" alt="{\textstyle {\hat {Y}}}" loading="lazy"> above must refer to the raw output of the classifier, not the discrete prediction; for example, with an <a href="Artificial_neural_network" class="mw-redirect" title="Artificial neural network">artificial neural network</a> and a classification problem, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle {\hat {Y}}}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle {\hat {Y}}}</annotation>

</semantics>

</math><img src="./d62b45891b318ecf6599faa7421e504490df672a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.676ex;" alt="{\textstyle {\hat {Y}}}" loading="lazy"> could refer to the output of the <a href="Softmax_function" title="Softmax function">softmax layer</a>.

Then we update <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle U}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle U}</annotation>

</semantics>

</math><img src="./087a7d8a39fe35012cbf2f561879f9e975cb4555.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.783ex; height:2.176ex;" alt="{\textstyle U}" loading="lazy"> to minimize <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle L_{A}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle L_{A}}</annotation>

</semantics>

</math><img src="./1d527ec10385f86a6befce2f46b3b264709f905a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.048ex; height:2.509ex;" alt="{\textstyle L_{A}}" loading="lazy"> at each training step according to the <a href="Gradient" title="Gradient">gradient</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \nabla _{U}L_{A}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \nabla _{U}L_{A}}</annotation>

</semantics>

</math><img src="./e5017f168456f20aa431a1e4cc894ebc7257a09c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:6.476ex; height:2.509ex;" alt="{\textstyle \nabla _{U}L_{A}}" loading="lazy"> and we modify <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle W}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle W}</annotation>

</semantics>

</math><img src="./e95737ee2530885a10b104e9cd5331077e1c88d5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.435ex; height:2.176ex;" alt="{\textstyle W}" loading="lazy"> according to the expression:

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \nabla _{W}L_{P}-proj_{\nabla _{W}L_{A}}\nabla _{W}L_{P}-\alpha \nabla _{W}L_{A}}</annotation>

</semantics>

</math>

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="\alpha ">

<annotation encoding="application/x-tex">\alpha</annotation>

</semantics>

</math><img src="./ee2a319db178e51919baafc482b83b8a29411715.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.488ex; height:1.676ex;" alt="\alpha " loading="lazy"> is a tunable <a href="Hyperparameter_optimization" title="Hyperparameter optimization">hyperparameter</a> that can vary at each time step.

The intuitive idea is that we want the predictor to try to minimize <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle L_{P}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle L_{P}}</annotation>

</semantics>

</math><img src="./ae3625acfc2bba8e5a00383bbfca190b4cc3d843.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.049ex; height:2.509ex;" alt="{\textstyle L_{P}}" loading="lazy"> (therefore the term <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \nabla _{W}L_{P}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \nabla _{W}L_{P}}</annotation>

</semantics>

</math><img src="./f0c0dad6b81f57505a8c19d6b802591de4514076.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:6.939ex; height:2.509ex;" alt="{\textstyle \nabla _{W}L_{P}}" loading="lazy">) while, at the same time, maximize <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle L_{A}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle L_{A}}</annotation>

</semantics>

</math><img src="./1d527ec10385f86a6befce2f46b3b264709f905a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.048ex; height:2.509ex;" alt="{\textstyle L_{A}}" loading="lazy"> (therefore the term <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle -\alpha \nabla _{W}L_{A}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle -\alpha \nabla _{W}L_{A}}</annotation>

</semantics>

</math><img src="./e899664c00215b770f0f7e1d51ac964a0c067d12.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:10.234ex; height:2.509ex;" alt="{\textstyle -\alpha \nabla _{W}L_{A}}" loading="lazy">), so that the adversary fails at predicting the sensitive variable from <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle {\hat {Y}}}">

<mover>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle {\hat {Y}}}</annotation>

</semantics>

</math><img src="./d62b45891b318ecf6599faa7421e504490df672a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.171ex; width:1.773ex; height:2.676ex;" alt="{\textstyle {\hat {Y}}}" loading="lazy">.

The term <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle -proj_{\nabla _{W}L_{A}}\nabla _{W}L_{P}}">

<msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle -proj_{\nabla _{W}L_{A}}\nabla _{W}L_{P}}</annotation>

</semantics>

</math><img src="./a3ee085efd45df762d03c40cdc3a1de9f4844fc6.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:18.499ex; height:2.843ex;" alt="{\textstyle -proj_{\nabla _{W}L_{A}}\nabla _{W}L_{P}}" loading="lazy"> prevents the predictor from moving in a direction that helps the adversary decrease its loss function.

It can be shown that training a predictor classification model with this algorithm improves <a href="#Definitions_based_on_predicted_outcome">demographic parity</a> with respect to training it without the adversary.

<div class="mw-heading mw-heading3"><h3 id="Postprocessing">Postprocessing</h3></div>

The final method tries to correct the results of a classifier to achieve fairness. In this method, we have a classifier that returns a score for each individual and we need to do a binary prediction for them. High scores are likely to get a positive outcome, while low scores are likely to get a negative one, but we can adjust the threshold to determine when to answer yes as desired. Note that variations in the threshold value affect the trade-off between the rates for true positives and true negatives.

If the score function is fair in the sense that it is independent of the protected attribute, then any choice of the threshold will also be fair, but classifiers of this type tend to be biased, so a different threshold may be required for each protected group to achieve fairness.<a href="#cite_note-hardt-53">[53]</a> A way to do this is plotting the true positive rate against the false negative rate at various threshold settings (this is called <a href="ROC_curve" class="mw-redirect" title="ROC curve">ROC curve</a>) and find a threshold where the rates for the protected group and other individuals are equal.<a href="#cite_note-hardt-53">[53]</a>

<div class="mw-heading mw-heading4"><h4 id="Reject_option_based_classification">Reject option based classification</h4></div>

Given a <a href="Statistical_classification" title="Statistical classification">classifier</a> let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle P(+|X)}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle P(+|X)}</annotation>

</semantics>

</math><img src="./5fe258e425ce38800ac6c33a357fcbe0dc01b292.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.99ex; height:2.843ex;" alt="{\textstyle P(+|X)}" loading="lazy"> be the probability computed by the classifiers as the <a href="Probability" title="Probability">probability</a> that the instance <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X}</annotation>

</semantics>

</math><img src="./8d80c41192705e1a6c6de1d65e16d7f70fbac391.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\textstyle X}" loading="lazy"> belongs to the positive class +. When <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle P(+|X)}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle P(+|X)}</annotation>

</semantics>

</math><img src="./5fe258e425ce38800ac6c33a357fcbe0dc01b292.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.99ex; height:2.843ex;" alt="{\textstyle P(+|X)}" loading="lazy"> is close to 1 or to 0, the instance <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X}</annotation>

</semantics>

</math><img src="./8d80c41192705e1a6c6de1d65e16d7f70fbac391.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\textstyle X}" loading="lazy"> is specified with high degree of certainty to belong to class + or – respectively. However, when <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle P(+|X)}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle P(+|X)}</annotation>

</semantics>

</math><img src="./5fe258e425ce38800ac6c33a357fcbe0dc01b292.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.99ex; height:2.843ex;" alt="{\textstyle P(+|X)}" loading="lazy"> is closer to 0.5 the classification is more unclear.<a href="#cite_note-roc-54">[54]</a>

We say <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X}</annotation>

</semantics>

</math><img src="./8d80c41192705e1a6c6de1d65e16d7f70fbac391.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\textstyle X}" loading="lazy"> is a "rejected instance" if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle max(P(+|X),1-P(+|X))\leq \theta }">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle max(P(+|X),1-P(+|X))\leq \theta }</annotation>

</semantics>

</math><img src="./d0acecc67bc592550456d7876969d6b1db66225b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:31.614ex; height:2.843ex;" alt="{\textstyle max(P(+|X),1-P(+|X))\leq \theta }" loading="lazy"> with a certain <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \theta }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \theta }</annotation>

</semantics>

</math><img src="./a11744bd71a5eb6efe4f28e12ca57f874d82658c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.09ex; height:2.176ex;" alt="{\textstyle \theta }" loading="lazy"> such that <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle 0.5<\theta <1}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle 0.5<\theta <1}</annotation>

</semantics>

</math><img src="./1d3ffe744a7cd8fb77e24820226e27f522f48657.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:11.422ex; height:2.176ex;" alt="{\textstyle 0.5<\theta <1}" loading="lazy">.

The algorithm of "ROC" consists on classifying the non-rejected instances following the rule above and the rejected instances as follows: if the instance is an example of a deprived group (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X(A)=a}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X(A)=a}</annotation>

</semantics>

</math><img src="./7675612b8bbf7a8c175fad9948a80d5335058fac.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:9.861ex; height:2.843ex;" alt="{\displaystyle X(A)=a}" loading="lazy">) then label it as positive, otherwise, label it as negative.

We can optimize different measures of discrimination (link) as functions of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \theta }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \theta }</annotation>

</semantics>

</math><img src="./a11744bd71a5eb6efe4f28e12ca57f874d82658c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.09ex; height:2.176ex;" alt="{\textstyle \theta }" loading="lazy"> to find the optimal <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \theta }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \theta }</annotation>

</semantics>

</math><img src="./a11744bd71a5eb6efe4f28e12ca57f874d82658c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.09ex; height:2.176ex;" alt="{\textstyle \theta }" loading="lazy"> for each problem and avoid becoming discriminatory against the privileged group.<a href="#cite_note-roc-54">[54]</a>

<ul><li><a href="Algorithmic_bias" title="Algorithmic bias">Algorithmic bias</a></li>

<li><a href="Machine_learning" title="Machine learning">Machine learning</a></li>

<li><a href="Representational_harm" title="Representational harm">Representational harm</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFCatonHaas2020" class="citation arxiv cs1">Caton, Simon; Haas, Christian (4 October 2020). "Fairness in Machine Learning: A Survey". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2010.04053">2010.04053</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-:0-2">^ <a href="#cite_ref-:0_2-0">a</a> <a href="#cite_ref-:0_2-1">b</a> <cite id="CITEREFMattu" class="citation web cs1">Mattu, Julia Angwin, Jeff Larson, Lauren Kirchner, Surya. <a rel="nofollow" class="external text" href="https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing">"Machine Bias"</a>. ProPublica. Retrieved 16 April 2022.</cite><code class="cs1-code">{{cite web}}</code>: CS1 maint: multiple names: authors list (link)

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> <cite id="CITEREFFriedlerScheideggerVenkatasubramanian2021" class="citation journal cs1">Friedler, Sorelle A.; Scheidegger, Carlos; Venkatasubramanian, Suresh (April 2021). <a rel="nofollow" class="external text" href="https://dl.acm.org/doi/10.1145/3433949">"The (Im)possibility of fairness: different value systems require different mechanisms for fair decision making"</a>. Communications of the ACM. 64 (4): 136–143. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3433949">10.1145/3433949</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0001-0782">0001-0782</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:1769114">1769114</a>.</cite>

</li>

<li id="cite_note-4"><a href="#cite_ref-4">^</a> <cite id="CITEREFMehrabiMorstatterSaxenaLerman2021" class="citation journal cs1">Mehrabi, Ninareh; Morstatter, Fred; Saxena, Nripsuta; Lerman, Kristina; Galstyan, Aram (13 July 2021). <a rel="nofollow" class="external text" href="https://doi.org/10.1145/3457607">"A Survey on Bias and Fairness in Machine Learning"</a>. ACM Computing Surveys. 54 (6): 115:1–115:35. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1908.09635">1908.09635</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3457607">10.1145/3457607</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0360-0300">0360-0300</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:201666566">201666566</a>.</cite>

</li>

<li id="cite_note-IBM-5"><a href="#cite_ref-IBM_5-0">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://web.archive.org/web/20220629170411/https://aif360.mybluemix.net/">"AI Fairness 360"</a>. aif360.mybluemix.net. Archived from <a rel="nofollow" class="external text" href="https://aif360.mybluemix.net/">the original</a> on 29 June 2022. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-6"><a href="#cite_ref-6">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.techrepublic.com/google-amp/article/ibm-ai-fairness-360-open-source-toolkit-adds-new-functionalities/">"IBM AI Fairness 360 open source toolkit adds new functionalities"</a>. Tech Republic. 4 June 2020.</cite>

</li>

<li id="cite_note-7"><a href="#cite_ref-7">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://ai.google/responsibilities/responsible-ai-practices/">"Responsible AI practices"</a>. Google AI. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-8"><a href="#cite_ref-8">^</a> <cite class="citation cs2"><a rel="nofollow" class="external text" href="https://github.com/tensorflow/fairness-indicators">Fairness Indicators</a>, tensorflow, 10 November 2022, retrieved 18 November 2022</cite>

</li>

<li id="cite_note-9"><a href="#cite_ref-9">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://ai.facebook.com/blog/how-were-using-fairness-flow-to-help-build-ai-that-works-better-for-everyone/">"How we're using Fairness Flow to help build AI that works better for everyone"</a>. ai.facebook.com. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-10"><a href="#cite_ref-10">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://venturebeat.com/business/ai-experts-warn-facebooks-anti-bias-tool-is-completely-insufficient/">"AI experts warn Facebook's anti-bias tool is 'completely insufficient'"</a>. VentureBeat. 31 March 2021. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-Hutchinson_Mitchell_2019_p.-11"><a href="#cite_ref-Hutchinson_Mitchell_2019_p._11-0">^</a> <cite id="CITEREFHutchinsonMitchell2019" class="citation conference cs1">Hutchinson, Ben; Mitchell, Margaret (29 January 2019). "50 Years of Test (Un)fairness". Proceedings of the Conference on Fairness, Accountability, and Transparency. New York, NY, USA: ACM FAT*'19. pp. 49–58. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1811.10104">1811.10104</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3287560.3287600">10.1145/3287560.3287600</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781450361255</bdi>.</cite>

</li>

<li id="cite_note-:1-12">^ <a href="#cite_ref-:1_12-0">a</a> <a href="#cite_ref-:1_12-1">b</a> <a href="#cite_ref-:1_12-2">c</a> <cite id="CITEREFLuoPuettSmith2023" class="citation cs2">Luo, Queenie; Puett, Michael J.; Smith, Michael D. (23 May 2023), A Perspectival Mirror of the Elephant: Investigating Language Bias on Google, ChatGPT, Wikipedia, and YouTube, <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2303.16281">2303.16281</a></cite>

</li>

<li id="cite_note-13"><a href="#cite_ref-13">^</a> <cite id="CITEREFKotekDockumSun2023" class="citation book cs1">Kotek, Hadas; Dockum, Rikker; Sun, David (5 November 2023). <a rel="nofollow" class="external text" href="https://dl.acm.org/doi/10.1145/3582269.3615599">"Gender bias and stereotypes in Large Language Models"</a>. Proceedings of the ACM Collective Intelligence Conference. CI '23. New York, NY, USA: Association for Computing Machinery. pp. 12–24. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3582269.3615599">10.1145/3582269.3615599</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>979-8-4007-0113-9</bdi>.</cite>

</li>

<li id="cite_note-14"><a href="#cite_ref-14">^</a> <cite id="CITEREFZhouTan2023" class="citation journal cs1">Zhou, Karen; Tan, Chenhao (December 2023). Bouamor, Houda; Pino, Juan; Bali, Kalika (eds.). <a rel="nofollow" class="external text" href="https://aclanthology.org/2023.findings-emnlp.696">"Entity-Based Evaluation of Political Bias in Automatic Summarization"</a>. Findings of the Association for Computational Linguistics: EMNLP 2023. Singapore: Association for Computational Linguistics: 10374–10386. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2305.02321">2305.02321</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.18653%2Fv1%2F2023.findings-emnlp.696">10.18653/v1/2023.findings-emnlp.696</a>.</cite>

</li>

<li id="cite_note-15"><a href="#cite_ref-15">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.justice.gov/opa/speech/attorney-general-eric-holder-speaks-national-association-criminal-defense-lawyers-57th">"Attorney General Eric Holder Speaks at the National Association of Criminal Defense Lawyers 57th Annual Meeting and 13th State Criminal Justice Network Conference"</a>. www.justice.gov. 1 August 2014. Retrieved 16 April 2022.</cite>

</li>

<li id="cite_note-16"><a href="#cite_ref-16">^</a> <cite id="CITEREFDieterichMendozaBrennan2016" class="citation journal cs1">Dieterich, William; Mendoza, Christina; Brennan, Tim (2016). <a rel="nofollow" class="external text" href="https://njoselson.github.io/pdfs/ProPublica_Commentary_Final_070616.pdf">"COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity"</a> (PDF). Northpointe Inc.</cite>

</li>

<li id="cite_note-17"><a href="#cite_ref-17">^</a> <cite id="CITEREFAngwin2016" class="citation web cs1">Angwin, Jeff Larson, Julia (29 July 2016). <a rel="nofollow" class="external text" href="https://www.propublica.org/article/technical-response-to-northpointe">"Technical Response to Northpointe"</a>. ProPublica. Retrieved 18 November 2022.</cite><code class="cs1-code">{{cite web}}</code>: CS1 maint: multiple names: authors list (link)

</li>

<li id="cite_note-18"><a href="#cite_ref-18">^</a> <cite id="CITEREFRose2010" class="citation magazine cs1">Rose, Adam (22 January 2010). <a rel="nofollow" class="external text" href="http://content.time.com/time/business/article/0,8599,1954643,00.html">"Are face-detection cameras racist?"</a>. Time. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0040-781X">0040-781X</a>. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-19"><a href="#cite_ref-19">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.theguardian.com/technology/2015/jul/01/google-sorry-racist-auto-tag-photo-app">"Google says sorry for racist auto-tag in photo app"</a>. The Guardian. 1 July 2015. Retrieved 16 April 2022.</cite>

</li>

<li id="cite_note-20"><a href="#cite_ref-20">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.theguardian.com/technology/2016/sep/08/artificial-intelligence-beauty-contest-doesnt-like-black-people">"A beauty contest was judged by AI and the robots didn't like dark skin"</a>. The Guardian. 8 September 2016. Retrieved 16 April 2022.</cite>

</li>

<li id="cite_note-Buolamwini2018-21">^ <a href="#cite_ref-Buolamwini2018_21-0">a</a> <a href="#cite_ref-Buolamwini2018_21-1">b</a> <cite id="CITEREFBuolamwiniGebru2018" class="citation conference cs1"><a href="Joy_Buolamwini" title="Joy Buolamwini">Buolamwini, Joy</a>; <a href="Timnit_Gebru" title="Timnit Gebru">Gebru, Timnit</a> (February 2018). <a rel="nofollow" class="external text" href="http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf">Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification</a> (PDF). Conference on Fairness, Accountability and Transparency. New York, NY, USA. pp. 77–91.</cite>

</li>

<li id="cite_note-22"><a href="#cite_ref-22">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.theguardian.com/technology/2021/aug/10/twitters-image-cropping-algorithm-prefers-younger-slimmer-faces-with-lighter-skin-analysis">"Student proves Twitter algorithm 'bias' toward lighter, slimmer, younger faces"</a>. The Guardian. 10 August 2021. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-23"><a href="#cite_ref-23">^</a> <cite class="citation cs2"><a rel="nofollow" class="external text" href="https://github.com/openai/dalle-2-preview/blob/eeec5a1843b1d17cb9ed113117a2fcaa9206a564/system-card.md">openai/dalle-2-preview</a>, OpenAI, 17 November 2022, retrieved 18 November 2022</cite>

</li>

<li id="cite_note-24"><a href="#cite_ref-24">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.nbcnews.com/tech/tech-news/no-quick-fix-openais-dalle-2-illustrated-challenges-bias-ai-rcna39918">"No quick fix: How OpenAI's DALL·E 2 illustrated the challenges of bias in AI"</a>. NBC News. 27 July 2022. Retrieved 23 July 2024.</cite>

</li>

<li id="cite_note-25"><a href="#cite_ref-25">^</a> <cite class="citation news cs1"><a rel="nofollow" class="external text" href="https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G">"Amazon scraps secret AI recruiting tool that showed bias against women"</a>. Reuters. 10 October 2018. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-26"><a href="#cite_ref-26">^</a> <cite class="citation news cs1"><a rel="nofollow" class="external text" href="https://www.washingtonpost.com/business/2019/11/11/apple-card-algorithm-sparks-gender-bias-allegations-against-goldman-sachs/">"Apple Card algorithm sparks gender bias allegations against Goldman Sachs"</a>. Washington Post. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0190-8286">0190-8286</a>. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-27"><a href="#cite_ref-27">^</a> <cite id="CITEREFMartinezKirchner2021" class="citation web cs1">Martinez, Emmanuel; Kirchner, Lauren (25 August 2021). <a rel="nofollow" class="external text" href="https://themarkup.org/denied/2021/08/25/the-secret-bias-hidden-in-mortgage-approval-algorithms">"The Secret Bias Hidden in Mortgage-Approval Algorithms – The Markup"</a>. themarkup.org. Retrieved 18 November 2022.</cite>

</li>

<li id="cite_note-Ruggieri_Alvarez_Pugnana_State_2023_pp._15421–15430-28"><a href="#cite_ref-Ruggieri_Alvarez_Pugnana_State_2023_pp._15421–15430_28-0">^</a> <cite id="CITEREFRuggieriAlvarezPugnanaState2023" class="citation journal cs1">Ruggieri, Salvatore; Alvarez, Jose M.; Pugnana, Andrea; State, Laura; Turini, Franco (26 June 2023). <a rel="nofollow" class="external text" href="https://doi.org/10.1609%2Faaai.v37i13.26798">"Can We Trust Fair-AI?"</a>. Proceedings of the AAAI Conference on Artificial Intelligence. 37 (13). Association for the Advancement of Artificial Intelligence (AAAI): 15421–15430. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1609%2Faaai.v37i13.26798">10.1609/aaai.v37i13.26798</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<a rel="nofollow" class="external text" href="https://hdl.handle.net/11384%2F136444">11384/136444</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/2374-3468">2374-3468</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:259678387">259678387</a>.</cite>

</li>

<li id="cite_note-Buyl_De_Bie_p.-29"><a href="#cite_ref-Buyl_De_Bie_p._29-0">^</a> <cite id="CITEREFBuylDe_Bie2022" class="citation journal cs1">Buyl, Maarten; De Bie, Tijl (2022). "Inherent Limitations of AI Fairness". Communications of the ACM. 67 (2): 48–55. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2212.06495">2212.06495</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3624700">10.1145/3624700</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<a rel="nofollow" class="external text" href="https://hdl.handle.net/1854%2FLU-01GMNH04RGNVWJ730BJJXGCY99">1854/LU-01GMNH04RGNVWJ730BJJXGCY99</a>.</cite>

</li>

<li id="cite_note-Castelnovo_Inverardi_Nanino_Penco_p.-30"><a href="#cite_ref-Castelnovo_Inverardi_Nanino_Penco_p._30-0">^</a> <cite id="CITEREFCastelnovoInverardiNaninoPenco2023" class="citation arxiv cs1">Castelnovo, Alessandro; Inverardi, Nicole; Nanino, Gabriele; Penco, Ilaria Giuseppina; Regoli, Daniele (2023). "Fair Enough? A map of the current limitations of the requirements to have "fair" algorithms". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2311.12435">2311.12435</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.AI">cs.AI</a>].</cite>

</li>

<li id="cite_note-31"><a href="#cite_ref-31">^</a> <cite id="CITEREFGaudeulArrigoniCharisiEscobar-Planas2024" class="citation cs2">Gaudeul, Alexia; Arrigoni, Ottla; Charisi, Vicky; Escobar-Planas, Marina; Hupont, Isabelle (2024), <a rel="nofollow" class="external text" href="https://ebooks.iospress.nl/doi/10.3233/FAIA240598">"Understanding the Impact of Human Oversight on Discriminatory Outcomes in AI-Supported Decision-Making"</a>, ECAI 2024, Frontiers in Artificial Intelligence and Applications, IOS Press, pp. 1067–1074, <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.3233%2Ffaia240598">10.3233/faia240598</a>, <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-64368-548-9</bdi>, retrieved 24 January 2025</cite>

</li>

<li id="cite_note-Barocas-32">^ <a href="#cite_ref-Barocas_32-0">a</a> <a href="#cite_ref-Barocas_32-1">b</a> <a href="#cite_ref-Barocas_32-2">c</a> <a href="#cite_ref-Barocas_32-3">d</a> Solon Barocas; Moritz Hardt; Arvind Narayanan, <a rel="nofollow" class="external text" href="http://www.fairmlbook.org">Fairness and Machine Learning</a>. Retrieved 15 December 2019.

</li>

<li id="cite_note-33"><a href="#cite_ref-33">^</a> <cite id="CITEREFMayes2001" class="citation book cs1">Mayes, Elizabeth (2001). Handbook of Credit Scoring. NY, NY, USA: Glenlake Publishing. p. 282. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-8144-0619-X</bdi>.</cite>

</li>

<li id="cite_note-34"><a href="#cite_ref-34">^</a> <cite id="CITEREFBerkHeidariJabbariKearns2021" class="citation journal cs1">Berk, Richard; Heidari, Hoda; Jabbari, Shahin; Kearns, Michael; Roth, Aaron (February 2021). <a rel="nofollow" class="external text" href="http://journals.sagepub.com/doi/10.1177/0049124118782533">"Fairness in Criminal Justice Risk Assessments: The State of the Art"</a>. Sociological Methods & Research. 50 (1): 3–44. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1703.09207">1703.09207</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1177%2F0049124118782533">10.1177/0049124118782533</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0049-1241">0049-1241</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:12924416">12924416</a>.</cite>

</li>

<li id="cite_note-Räz_129–137-35">^ <a href="#cite_ref-Räz_129–137_35-0">a</a> <a href="#cite_ref-Räz_129–137_35-1">b</a> <cite id="CITEREFRäz2021" class="citation book cs1">Räz, Tim (3 March 2021). <a rel="nofollow" class="external text" href="https://dl.acm.org/doi/10.1145/3442188.3445876">"Group Fairness: Independence Revisited"</a>. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. ACM. pp. 129–137. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2101.02968">2101.02968</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3442188.3445876">10.1145/3442188.3445876</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-4503-8309-7</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:231667399">231667399</a>.</cite>

</li>

<li id="cite_note-metrics_paper-36">^ <a href="#cite_ref-metrics_paper_36-0">a</a> <a href="#cite_ref-metrics_paper_36-1">b</a> <a href="#cite_ref-metrics_paper_36-2">c</a> <cite id="CITEREFVermaRubin2018" class="citation book cs1">Verma, Sahil; Rubin, Julia (2018). <a rel="nofollow" class="external text" href="https://dl.acm.org/doi/10.1145/3194770.3194776">"Fairness definitions explained"</a>. Proceedings of the International Workshop on Software Fairness. pp. 1–7. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3194770.3194776">10.1145/3194770.3194776</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781450357463</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:49561627">49561627</a>.</cite>

</li>

<li id="cite_note-37"><a href="#cite_ref-37">^</a> <cite id="CITEREFGursoyKakadiaris2022" class="citation book cs1">Gursoy, Furkan; Kakadiaris, Ioannis A. (November 2022). <a rel="nofollow" class="external text" href="https://ieeexplore.ieee.org/document/10029385">"Equal Confusion Fairness: Measuring Group-Based Disparities in Automated Decision Systems"</a>. 2022 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE. pp. 137–146. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2307.00472">2307.00472</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FICDMW58026.2022.00027">10.1109/ICDMW58026.2022.00027</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>979-8-3503-4609-1</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:256669476">256669476</a>.</cite>

</li>

<li id="cite_note-chen-hooker-2021-38"><a href="#cite_ref-chen-hooker-2021_38-0">^</a> <cite id="CITEREFChenHooker2021" class="citation arxiv cs1">Chen, Violet (Xinying); Hooker, J. N. (2021). "Welfare-based Fairness through Optimization". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2102.00311">2102.00311</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.AI">cs.AI</a>].</cite>

</li>

<li id="cite_note-mullainathan-ec-2018-39"><a href="#cite_ref-mullainathan-ec-2018_39-0">^</a> <cite id="CITEREFMullainathan2018" class="citation audio-visual cs1"><a href="Sendhil_Mullainathan" title="Sendhil Mullainathan">Mullainathan, Sendhil</a> (19 June 2018). <a rel="nofollow" class="external text" href="https://www.youtube.com/watch?v=rp965fnd3qE">Algorithmic Fairness and the Social Welfare Function</a>. Keynote at the 19th ACM Conference on Economics and Computation (EC'18). YouTube. 48 minutes in. <q>In other words, if you have a social welfare function where what you care about is harm, and you care about harm to the African Americans, there you go: 12 percent less African Americans in jail overnight.... Before we get into the minutiae of relative harm, the welfare function is defined in absolute harm, so we should actually calculate the absolute harm first.</q></cite>

</li>

<li id="cite_note-mitchell2021-40"><a href="#cite_ref-mitchell2021_40-0">^</a> <cite id="CITEREFMitchellPotashBarocasd'Amour2021" class="citation journal cs1">Mitchell, Shira; Potash, Eric; Barocas, Solon; d'Amour, Alexander; Lum, Kristian (2021). <a rel="nofollow" class="external text" href="https://doi.org/10.1146%2Fannurev-statistics-042720-125902">"Algorithmic Fairness: Choices, Assumptions, and Definitions"</a>. Annual Review of Statistics and Its Application. 8 (1): 141–163. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1811.07867">1811.07867</a>. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2021AnRSA...8..141M">2021AnRSA...8..141M</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1146%2Fannurev-statistics-042720-125902">10.1146/annurev-statistics-042720-125902</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:228893833">228893833</a>.</cite>

</li>

<li id="cite_note-castelnovo2022-41"><a href="#cite_ref-castelnovo2022_41-0">^</a> <cite id="CITEREFCastelnovoCrupiGrecoRegoli2022" class="citation journal cs1">Castelnovo, Alessandro; Crupi, Riccardo; Greco, Greta; Regoli, Daniele; Penco, Ilaria Giuseppina; Cosentini, Andrea Claudio (2022). <a rel="nofollow" class="external text" href="https://doi.org/10.1038/s41598-022-07939-1">"A clarification of the nuances in the fairness metrics landscape"</a>. Scientific Reports. 12 (1): 4209. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2106.00467">2106.00467</a>. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2022NatSR..12.4209C">2022NatSR..12.4209C</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1038%2Fs41598-022-07939-1">10.1038/s41598-022-07939-1</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8913820">8913820</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/35273279">35273279</a>.</cite>

</li>

<li id="cite_note-mehrabi2021-42"><a href="#cite_ref-mehrabi2021_42-0">^</a> Mehrabi, Ninareh, Fred Morstatter, Nripsuta Saxena, <a href="Kristina_Lerman" title="Kristina Lerman">Kristina Lerman</a>, and Aram Galstyan. <a rel="nofollow" class="external text" href="https://doi.org/10.1145/3457607">"A survey on bias and fairness in machine learning."</a> ACM Computing Surveys (CSUR) 54, no. 6 (2021): 1–35.

</li>

<li id="cite_note-43"><a href="#cite_ref-43">^</a> <cite id="CITEREFDworkHardtPitassiReingold2012" class="citation book cs1">Dwork, Cynthia; Hardt, Moritz; Pitassi, Toniann; Reingold, Omer; Zemel, Richard (2012). <a rel="nofollow" class="external text" href="https://doi.org/10.1145/2090236.2090255">"Fairness through awareness"</a>. Proceedings of the 3rd Innovations in Theoretical Computer Science Conference on - ITCS '12. pp. 214–226. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F2090236.2090255">10.1145/2090236.2090255</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781450311151</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:13496699">13496699</a>.</cite>

</li>

<li id="cite_note-causal-44"><a href="#cite_ref-causal_44-0">^</a> <cite id="CITEREFGalhotraBrunMeliou2017" class="citation book cs1">Galhotra, Sainyam; Brun, Yuriy; Meliou, Alexandra (2017). "Fairness testing: Testing software for discrimination". Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. pp. 498–510. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1709.03221">1709.03221</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3106237.3106277">10.1145/3106237.3106277</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781450351058</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:6324652">6324652</a>.</cite>

</li>

<li id="cite_note-cff-45"><a href="#cite_ref-cff_45-0">^</a> Kusner, M. J., Loftus, J., Russell, C., & Silva, R. (2017). <a rel="nofollow" class="external text" href="https://proceedings.neurips.cc/paper/2017/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html">Counterfactual fairness</a>. Advances in neural information processing systems, 30.

</li>

<li id="cite_note-46"><a href="#cite_ref-46">^</a> <cite id="CITEREFCostonMishlerKennedyChouldechova2020" class="citation book cs1">Coston, Amanda; Mishler, Alan; Kennedy, Edward H.; Chouldechova, Alexandra (27 January 2020). "Counterfactual risk assessments, evaluation, and fairness". Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. FAT* '20. New York, NY, USA: Association for Computing Machinery. pp. 582–593. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3351095.3372851">10.1145/3351095.3372851</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-4503-6936-7</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:202539649">202539649</a>.</cite>

</li>

<li id="cite_note-47"><a href="#cite_ref-47">^</a> <cite id="CITEREFMishlerKennedyChouldechova2021" class="citation book cs1">Mishler, Alan; Kennedy, Edward H.; Chouldechova, Alexandra (1 March 2021). "Fairness in Risk Assessment Instruments". Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. FAccT '21. New York, NY, USA: Association for Computing Machinery. pp. 386–400. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3442188.3445902">10.1145/3442188.3445902</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-4503-8309-7</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:221516412">221516412</a>.</cite>

</li>

<li id="cite_note-Plecko_Bareinboim_p.-48">^ <a href="#cite_ref-Plecko_Bareinboim_p._48-0">a</a> <a href="#cite_ref-Plecko_Bareinboim_p._48-1">b</a> <cite id="CITEREFPleckoBareinboim2022" class="citation journal cs1">Plecko, Drago; Bareinboim, Elias (2022). "Causal Fairness Analysis". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2207.11385">2207.11385</a>.</cite> <code class="cs1-code">{{cite journal}}</code>: Cite journal requires <code class="cs1-code">|journal=</code> (help)

</li>

<li id="cite_note-zemel-49"><a href="#cite_ref-zemel_49-0">^</a> Richard Zemel; Yu (Ledell) Wu; Kevin Swersky; Toniann Pitassi; Cyntia Dwork, <a rel="nofollow" class="external text" href="https://www.cs.toronto.edu/~toni/Papers/icml-final.pdf">Learning Fair Representations</a>. Retrieved 1 December 2019

</li>

<li id="cite_note-reweighing-50"><a href="#cite_ref-reweighing_50-0">^</a> Faisal Kamiran; Toon Calders, <a rel="nofollow" class="external text" href="https://link.springer.com/content/pdf/10.1007%2Fs10115-011-0463-8.pdf">Data preprocessing techniques for classification without discrimination</a>. Retrieved 17 December 2019

</li>

<li id="cite_note-zafar-51"><a href="#cite_ref-zafar_51-0">^</a> Muhammad Bilal Zafar; Isabel Valera; Manuel Gómez Rodríguez; Krishna P. Gummadi, <a rel="nofollow" class="external text" href="https://people.mpi-sws.org/~mzafar/papers/disparate_mistreatment.pdf">Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment</a>. Retrieved 1 December 2019

</li>

<li id="cite_note-adversarial1-52">^ <a href="#cite_ref-adversarial1_52-0">a</a> <a href="#cite_ref-adversarial1_52-1">b</a> Brian Hu Zhang; Blake Lemoine; Margaret Mitchell, <a rel="nofollow" class="external text" href="https://arxiv.org/abs/1801.07593">Mitigating Unwanted Biases with Adversarial Learning</a>. Retrieved 17 December 2019

</li>

<li id="cite_note-hardt-53">^ <a href="#cite_ref-hardt_53-0">a</a> <a href="#cite_ref-hardt_53-1">b</a> Moritz Hardt; Eric Price; Nathan Srebro, <a rel="nofollow" class="external text" href="https://arxiv.org/abs/1610.02413">Equality of Opportunity in Supervised Learning</a>. Retrieved 1 December 2019

</li>

<li id="cite_note-roc-54">^ <a href="#cite_ref-roc_54-0">a</a> <a href="#cite_ref-roc_54-1">b</a> Faisal Kamiran; Asim Karim; Xiangliang Zhang, <a rel="nofollow" class="external text" href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.722.3030&rep=rep1&type=pdf">Decision Theory for Discrimination-aware Classification</a>. Retrieved 17 December 2019

</li>

</ol></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-06-23" href="https://en.wikipedia.org/wiki/?title=Fairness_(machine_learning)&oldid=1297032712">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>